Image-capturing apparatus, image processing method, and program

ABSTRACT

The present technology relates to, for example, an image-capturing apparatus, an image processing method, and a program that make it possible to suppress an increase in costs.A controller performs an extraction control for extracting a first image and a second image from a captured image captured by an image sensor that performs image-capturing, the first image being displayed on a first display section, the second image being displayed on a second display section. A data amount adjuster adjusts data amounts of the first image and the second image according to vehicle information acquired from a vehicle. For example, the present technology is applicable to a viewing system that displays an image of a region behind a vehicle.

TECHNICAL FIELD

The present technology relates to an image-capturing apparatus, an imageprocessing method, and a program. In particular, the present technologyrelates to, for example, an image-capturing apparatus, an imageprocessing method, and a program that make it possible to suppress anincrease in costs.

BACKGROUND ART

For example, a viewing system in which a camera that is animage-capturing apparatus is installed in the rear of a vehicle such asan automobile and an image of a region behind the vehicle that iscaptured using the camera is displayed, is proposed.

001 Examples of the image of a region behind a vehicle that is providedby the viewing system include an image of a region situated furtherrearward than a region just behind the rear of the vehicle, and an imageof the region just behind the rear of the vehicle.

002 Here, the image of a region situated further rearward than a regionjust behind the rear of a vehicle is, for example, an image thatcorresponds to an image seen in a Class I mirror, which is a so-calledinterior rearview mirror, and is hereinafter also referred to as a backmirror (BM) image. Further, the image of a region just behind the rearof a vehicle is an image of the rear of the vehicle and a region justbehind the rear of the vehicle, and is hereinafter also referred to as arearview (RV) image.

003 The specifications, that is, angles of view and optical-axisdirections that are necessary to capture a BM image and an RV image aredifferent. Thus, the BM image and the RV image are respectively capturedusing different cameras. Therefore, in order to display a BM image andan RV image in the viewing system, there is a need to provide, in therear of a vehicle, two cameras that are a camera used to capture the BMimage and a camera used to capture the RV image.

004 In recent years, there is a need to capture a BM image and a RVimage using a single camera in order to, for example, reduce costs.

005 For example, Patent Literature 1 discloses a camera in which anangle of view and an optical-axis direction are changed by a PTZ (pan,tilt, and zoom) operation being performed. In the camera disclosed inPatent Literature 1, the PTZ operation makes it possible to switchbetween capturing of a distant image of a range distant from a vehicleand capturing of a nearby image of a range close to the vehicle.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Patent Application Laid-open No. 11-3123001

DISCLOSURE OF INVENTION Technical Problem

006 The camera disclosed in Patent Literature 1 makes it possible toswitch between output of a distant image corresponding to a BM image andoutput of a nearby image corresponding to an RV image. However, it isdifficult to simultaneously output the distant image and the nearbyimage using the camera disclosed in Patent Literature 1.

007 Further, there is a restriction on a vehicle transmission bandwidththat can be used to transmit data in a vehicle. Thus, even if it ispossible to simultaneously output a distant image corresponding to a BMimage and a nearby image corresponding to an RV image, it will bedifficult to simultaneously transmit the distant image and the nearbyimage to display those images.

008 Although an increase in vehicle transmission bandwidth makes itpossible to simultaneously transmit a distant image and a nearby image,the increase in vehicle transmission bandwidth results in increasing thecosts for the viewing system.

The present technology has been made in view of the circumstancesdescribed above, and is intended to make it possible to suppress anincrease in costs.

Solution to Problem

An image-capturing apparatus or a program according to the presenttechnology is an image-capturing apparatus, or a program that causes acomputer to operate as the image-capturing apparatus, theimage-capturing apparatus including an image sensor that performsimage-capturing; a controller that performs an extraction control forextracting a first image and a second image from a captured imagecaptured by the image sensor, the first image being displayed on a firstdisplay section, the second image being displayed on a second displaysection; and a data amount adjuster that adjusts data amounts of thefirst image and the second image according to vehicle informationacquired from a vehicle.

An image processing method according to the present technology includesperforming an extraction control for extracting a first image and asecond image from a captured image captured by an image sensor thatperforms image-capturing, the first image being displayed on a firstdisplay section, the second image being displayed on a second displaysection; and adjusting data amounts of the first image and the secondimage according to vehicle information acquired from a vehicle.

In an image-capturing apparatus, an image processing method, and aprogram according to the present technology, an extraction control forextracting a first image and a second image from a captured imagecaptured by an image sensor that performs image-capturing is performed,the first image being displayed on a first display section, the secondimage being displayed on a second display section. Then, data amounts ofthe first image and the second image are adjusted according to vehicleinformation acquired from a vehicle.

009 Note that the image-capturing apparatus may be an independentapparatus or an internal block included in a single apparatus.

0010 Further, the program can be provided by being transmitted through atransmission medium or by being stored in a recording medium.

0011

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a perspective view illustrating an external configurationexample of a vehicle 10 that includes a viewing system according to thepresent technology.

FIG. 2 is a perspective view illustrating an example of a configurationof the interior of the vehicle 10.

FIG. 3 is a block diagram illustrating a first example of aconfiguration of the viewing system included in the vehicle 10.

FIG. 4 is a diagram describing an example of a control of extraction ofa BM image and an RV image from a captured image, the control beingperformed by a controller 36.

FIG. 5 is a diagram describing the control of extraction of (a regionR11 that is) a BM image, the control being performed according togyroscopic information.

FIG. 6 illustrates an example of a configuration of an image sensor 32.

FIG. 7 illustrates an example of an image that can be output by theimage sensor 32.

FIG. 8 is a diagram describing an example of a vehicle transmissionbandwidth that can be used for data transmission in the vehicle 10.

FIG. 9 is a diagram describing a first example of a control ofadjustment of data amounts of a BM image and an RV image, the controlbeing performed by a controller 36.

FIG. 10 is a diagram describing a second example of the control ofadjustment of data amounts of a BM image and an RV image, the controlbeing performed by the controller 36.

FIG. 11 is a flowchart describing an example of display processing ofdisplaying a BM image and an RV image that is performed in the viewingsystem.

FIG. 12 is a block diagram illustrating a second example of theconfiguration of the viewing system included in the vehicle 10.

FIG. 13 is a block diagram illustrating a third example of theconfiguration of the viewing system included in the vehicle 10.

FIG. 14 is a diagram describing an example of a control of extraction ofa BM image and an RV image from a captured image, the control beingperformed by a controller 81.

FIG. 15 is a flowchart describing an example of display processing ofdisplaying a BM image and a RV image that is performed in the viewingsystem.

FIG. 16 is a block diagram illustrating a fourth example of theconfiguration of the viewing system included in the vehicle 10.

FIG. 17 is a block diagram illustrating an example of a configuration ofan embodiment of a computer to which the present technology is applied.

Mode(s) for Carrying Out the Invention

0012 <Example of Configuration of Vehicle Including Viewing System>

0013 FIG. 1 is a perspective view illustrating an example of aconfiguration of an appearance of a vehicle 10 that includes a viewingsystem to which the present technology is applied.

0014 For example, a camera unit 11 that is an image-capturing apparatusused to capture an image of a region behind the (four-wheeled) vehicle10 is installed in the rear of the vehicle 10. In FIG. 1, the cameraunit 11 is installed above a rear window of the vehicle 10.

0015 The camera unit 11 is a wide-angle camera unit such that it ispossible to capture an image in which both a range corresponding to a BMimage (a first image) and a range corresponding to an RV image (a secondimage) appear. Further, the camera unit 11 is a high-resolution cameraunit such that a distant subject is apparent in the BM image. Thus, thecamera unit 11 is capable of capturing a wide-angle, high-resolutionimage.

0016 Note that, in the camera unit 11, a BM image and an RV image areextracted from an image captured by the camera unit 11 (hereinafter alsoreferred to as a captured image). This will be described later.

0017 The camera unit 11 is installed in a state in which the orientationof the optical axis has been adjusted, such that the BM image includesan image of a state of a region situated further rearward than a regionjust behind the rear of the vehicle 10, and such that the RV imageincludes an image of states of the rear of the vehicle 10 and the regionjust behind the rear of the vehicle 10, the state of the region situatedfurther rearward than the region just behind the rear of the vehicle 10being a state that can be observed using an interior rearview mirror (aClass I mirror in Regulation No. 46 defined by the United NationsEconomic Commission for Europe <UNECE>) when the interior rearviewmirror is installed in the vehicle 10.

0018 Thus, the BM image is an image of a state of a region situatedfurther rearward than a region just behind the rear of the vehicle 10,the state being a state that can be observed using an interior rearviewmirror when the interior rearview mirror is installed in the vehicle 10.Further, the RV image is an image of states of the rear of the vehicle10 and the region just behind the rear of the vehicle 10. The RV imageis particularly useful when the vehicle 10 is traveling backward, sincea region just behind the rear of the vehicle 10 that is a blind spot ofthe interior rearview mirror appears in the RV image. Further, the RVimage can be used to generate an overhead image obtained when thevehicle 10 is viewed from above.

0019 Note that the camera unit 11 is not limited to being installedabove the rear window of the vehicle 10 as long as it is possible tocapture a captured image from which the BM image and the RV imagedescribed above can be extracted. For example, in addition to beinginstalled above the rear window of the vehicle 10, the camera unit 11may be installed at, for example, a position P11 above a license platesituated in the rear of the vehicle 10.

0020 FIG. 2 is a perspective view illustrating an example of aconfiguration of the interior of the vehicle 10 of FIG. 1.

0021 A BM display section 21 that displays thereon a BM image isprovided at a position, in the vehicle 10, at which the interiorrearview mirror is installed. The BM display section 21 is a displaysection that is an alternative to the interior rearview mirror.

0022 An RV display section 22 that displays thereon an RV image isprovided at a center position of a dashboard in the vehicle 10.

0023 Note that an in-vehicle camera 23 used to capture an image of adriver is provided on the side of a driver's seat of the dashboard inthe vehicle 10. An image of the driver is captured to be output by thein-vehicle camera 23. In the vehicle 10, positions of the line of sightand the head of the driver are detected from the image of the driver.

0024 Here, the in-vehicle camera 23 used to capture an image of thedriver may be provided at any position other than a position on thedashboard, such as a position P21 above the BM display section 21.

0025 <First Example of Configuration of Viewing System>

0026 FIG. 3 is a block diagram illustrating a first example of aconfiguration of the viewing system included in the vehicle 10.

0027 The viewing system includes the camera unit 11, the BM displaysection 21, and the RV display section 22 described with reference toFIGS. 1 and 2.

0028 The camera unit 11 includes an optical system 31, an image sensor32, a data amount adjuster 33, an output section 34, an acquisitionsection 35, and a controller 36.

0029 The optical system 31 includes optical components such as acondenser and a diaphragm, and collects light entering the opticalsystem 31 onto the image sensor 32.

0030 The image sensor 32 receives light from the optical system 31 andperforms photoelectric conversion to capture a captured image. Then, inaccordance with control performed by the controller 36, the image sensor32 extracts the BM image and the RV image from the captured image tooutput the read images. The BM image and the RV image that are output bythe image sensor 32 is supplied to the data amount adjuster 33.

0031 In accordance with control performed by the controller 36, the dataamount adjuster 33 adjusts data amounts of the BM image and the RV imagethat are output by the image sensor 32, and supplies, to the outputsection 34, the BM image and the RV image of which the respective dataamounts have been adjusted.

The output section 34 is an output interface (IF) that transmits, to theoutside of the camera unit 11, the BM image and the RV image from thedata amount adjuster 33. The output section 34 transmits the BM image tothe BM display section 21, and transmits the RV image to the RV displaysection 22. The BM image from the output section 34 is displayed on theBM display section 21 in accordance with the specification of the BMdisplay section 21, and the RV image from the output section 34 isdisplayed on the RV display section 22 in accordance with thespecification of the RV display section 22. The output section 34 iscapable of performing a format conversion and other image processing onthe BM image and the RV image as necessary.

The acquisition section 35 acquires, from the vehicle 10, informationthat can be acquired from the vehicle 10 (hereinafter also referred toas vehicle information), and supplies the acquired vehicle informationto the 0032 controller 36.

Here, examples of the vehicle information include traveling information,the specifications of the BM display section 21 and the RV displaysection 22, positions of the line of sight and the head of a driver ofthe vehicle 10, and gyroscopic information.

0033 The traveling information is information that indicates a travelingstate of the vehicle 10, and specifically indicates a vehicle speed anda traveling direction (forward or backward). For example, it is possibleto acquire the vehicle speed from output of a speed sensor when thevehicle 10 includes the speed sensor. For example, it is possible toacquire the traveling direction from a state of the transmission. 0034

0035 For example, the specifications of the BM display section 21 andthe RV display section 22 are the resolution of the BM display section21 and the resolution of the RV display section 22, and can be acquiredfrom the BM display section 21 and the RV display section 22.

The positions of the line of sight and the head of a driver of thevehicle 10 are obtained from an image captured by the in-vehicle camera23. 0036

0037 The gyroscopic information is information that indicates a pose ofthe vehicle 10 (/ an angle of the inclination of the vehicle). It ispossible to obtain the gyroscopic information from output of a gyroscopewhen the vehicle 10 includes the gyroscope. The use of the gyroscopicinformation makes it possible to recognize whether the vehicle 10 is ona hill.

The controller 36 controls the image sensor 32 and the data amountadjuster 33 according to the vehicle information supplied by theacquisition section 35.

0038 In other words, for example, the controller 36 performs, accordingto the vehicle information, an extraction control for controllingextraction of a BM image and an RV image from a captured image capturedby the image sensor 32. Further, according to the vehicle information,the controller 36 performs an adjustment control for controllingadjustment of data amounts of the BM image and the RV image that isperformed by the data amount adjuster 33.

0039 Thus, it can be said that the image sensor 32 extracts a BM imageand an RV image from a captured image according to vehicle information,and the data amount adjuster 33 adjusts data amounts of the BM image andthe RV image according to the vehicle information.

<Control of Reading of Read Image from Captured Image>

0040 FIG. 4 is a diagram describing an example of a control ofextraction of a BM image and an RV image from a captured image, thecontrol being performed by the controller 36.

0041 Regarding a positional relationship between the optical system 31and a captured image captured by the image sensor 32, the captured image(a light-receiving surface of the image sensor 32) includes an imagecircle of (a lens included in) the optical system 31, as illustrated in,for example, FIG. 4.

0042 In the extraction control, the controller 36 controls reading ofdata (a pixel signal) from the image sensor 32 such that a specifiedregion R11 is extracted from a captured image as a BM image, thespecified region R11 being a region in which a region situated furtherrearward than a region just behind the rear of the vehicle 10 appears (aregion observed using an interior rearview mirror if the interiorrearview mirror is installed in the vehicle 10).

0043 Further, in the extraction control, the controller 36 controlsreading of data from the image sensor 32 such that a specified regionR12 is extracted as an RV image from (an image in the image circle fromamong) the captured image, the specified region R12 being a region inwhich the rear of the vehicle 10 and the region just behind the rear ofthe vehicle 10 appear.

0044 In accordance with control performed by the controller 36, theimage sensor 32 reads, from a captured image obtained by performingimage-capturing, data (a pixel signal) of the region R11 thatcorresponds to (data of) the BM image, and reads, from the capturedimage, data of the region R12 that corresponds to the RV image.

0045 Note that the sizes of the regions R11 and R12 can be set accordingto the specifications of the BM display section 21 and the RV displaysection 22.

Further, in the control of extraction of a BM image, the controller 36controls (a position of) the region R11 extracted as the BM image,according to one of positions of the line of sight and the head of adriver, or according to both of them. 0046

In other words, if an interior rearview mirror is installed in thevehicle 10, a range that appears in an image that can be seen by adriver using the interior rearview mirror will be changed as the drivermoves his/her line of sight or his/her head. In the control ofextraction of a BM image, the controller 36 changes a position of theregion R11 extracted as the BM image, according to the positions of theline of sight and the head of the driver, such that the driver can see aBM image of a range similar to the range in an image that can beobserved when the interior rearview mirror is installed in the vehicle10.

0047 Moreover, in the control of extraction of a BM image, thecontroller 36 can control the region R11 extracted as the BM image,according to gyroscopic information included in vehicle information.

0048 FIG. 5 is a diagram describing the control of extraction of (theregion R11 that is) a BM image, the control being performed according togyroscopic information.

Here, the range of a three-dimensional space in a BM image is referredto as a BM range. Further, it is assumed that positions of the line ofsight and the head of a driver are fixed in order to simplify thedescription. Therefore, it is assumed that (a position of) the regionR11 extracted as a BM image from a captured image, and thus a BM rangeare not changed due to the driver moving the position of his/her line ofsight or his/her head.

0049 Further, it is assumed that, when the vehicle 10 is on a flat road,an object situated rearward a specified distance from the vehicle 10appears in a BM image.

0050 In this case, when the vehicle 10 is on an uphill, there is nochange in BM range, but the optical axis of the camera unit 11 installedin the rear of the vehicle 10 is downwardly inclined (toward the road),compared to when the vehicle 10 is on a flat road.

0051 Consequently, an object situated rearward a specified distance fromthe vehicle 10 that is within a BM image and appears in a BM image whenthe vehicle 10 is on a flat road, is not within the BM range and doesnot appears in the BM image when the vehicle 10 is on an uphill. Inother words, an object situated rearward a specified distance from thevehicle 10 appears in a BM image when the vehicle 10 is on a flat road,and does not appear in the BM image when the vehicle 10 is on an uphill.

0052 In this case, when the vehicle 10 is on an uphill, a rearwardrange, on a road, that can be confirmed by a driver of the vehicle 10using a BM image is smaller, compared to when the vehicle 10 is on aflat road.

0053 Thus, in the control of extraction of a BM image, the controller 36can control the region R11 extracted as the BM image, according togyroscopic information included in vehicle information.

0054 In other words, according to gyroscopic information, the controller36 can change the position of the region R11 extracted as a BM image toan upper position in a captured image when the front side of the vehicle10 is upwardly inclined.

0055 Accordingly, an object situated rearward a specified distance fromthe vehicle 10 appears in a BM image when the vehicle 10 is on a flatroad. Further, it is possible to prevent the object from not appearingin the BM image when the vehicle 10 is on an uphill. Note that,according to gyroscopic information, the controller 36 can change theposition of the region R11 extracted as a BM image to a lower positionin a captured image when the vehicle 10 is on a downhill.

0056 <Example of Configuration of Image Sensor 32>

FIG. 6 illustrates an example of a configuration of the image sensor 32of FIG. 3. 0057

0058 The image sensor 32 includes a plurality of pixels 51, ananalog-to-digital (AD) converter, and a line memory 53.

0059 The plurality of pixels 51 is arranged in a two-dimensional plane.In FIG. 6, the plurality of pixels 51 is arranged in a grid. A verticalsignal line (VSL) is routed with respect to each column of the pixels 51arranged in a grid, and the pixels 51 in each column are connected to aVSL routed with respect to the column. The pixel 51 converts lightentering the pixel 51 into a pixel signal that is an electrical signalcorresponding to an amount of the light. The pixel signal is read fromthe pixel 51 to be output to a VSL routed with respect to a column ofthat pixel 51.

0060 For example, an AD converter 52 includes, for each column of thepixels 51 arranged in a grid, an AD converter (ADC) (not illustrated)that is responsible for AD conversion performed on a pixel signal of thepixel 51 in the column. The ADC for each column is connected to a VSLrouted with respect to the column, and performs AD conversion on a pixelsignal read from the pixel 51 connected to the VSL. The pixel signal onwhich AD conversion has been performed by the ADC of the AD converter 52for each column is supplied to the line memory 53.

0061 The line memory 53 stores therein pixel signals of at most thepixels 51 included in one row and supplied by the AD converter 52, andsequentially transfers the stored pixel signals in a row direction tooutput the pixel signals to the outside of the image sensor 32.

0062 Here, in FIG. 6, the pixel 51 in the yth row from the top and inthe xth column from the left, and a pixel signal obtained from the pixel51 are also hereinafter respectively represented by a pixel yx and apixel signal yx as appropriate.

0063 A captured image is an image that is made up of pixel signalsobtained from all of the pixels (the effective pixels) 51 of thelight-receiving surface of the image sensor 32. In the image sensor 32,a pixel signal is read for each row in a direction of a lower row fromthe pixel 51 in an uppermost row a when the captured image is output.

0064 The pixel signals of the pixels 51 in one row are supplied to theAD converter 52 through a VSL. In the AD converter 52, AD conversion isperformed on the pixel signals of the pixels 51 in one row at the sametime, and the pixel signal on which AD conversion has been performed issupplied to be stored in the line memory 53.

0065 In the line memory 53, pixel signals of the pixels 51 in one rowthat are supplied by the AD converter 52 are stored and sequentiallytransferred in a row direction to be read into the outside of the imagesensor 32.

0066 The processing described above is performed with respect to all ofthe rows of the pixels 51, and this results in (pixel signals for) acaptured image for a single screen being read into the outside of theimage sensor 32.

0067 A BM image and an RV image correspond to (images of) regions thatare portions of a captured image. Thus, in order to obtain the BM imageand the RV image, there is no need to read the entire captured imagefrom the image sensor 32, and it is sufficient if only (pixel signalsfor) regions that are portions of the captured image that respectivelycorrespond to the BM image and the RV image, are read.

0068 Here, reading of an entire captured image from the image sensor 32is also referred to as full reading, and reading of a region that is aportion of the captured image from the image sensor 32 is also referredto as partial reading.

0069 In the extraction control for controlling reading of data from theimage sensor 32 such that a BM image and an RV image are extracted froma captured image, the image sensor 32 is controlled by the controller 36such that partial reading is performed.

0070 For example, when the region R11 corresponding to a BM image ismade up of the pixels 51 in columns 1 to 5 from among the pixels 51 inrows d to h, as illustrated in FIG. 6, the controller 36 controls theimage sensor 32 such that pixel signals of the pixels 51 in columns 1 to5 from among the pixels 51 in rows d to h are read.

0071 In this case, in the image sensor 32, the pixel signals of thepixels 51 in the row d are read, AD conversion is performed by the ADconverter 52 with respect to the read pixel signals, and the pixelsignals on which AD conversion has been performed are supplied to theline memory 53. Then, the line memory 53 stores therein the pixelsignals of the pixels 51 in the row d, the pixel signals being pixelsignals on which AD conversion has been performed. Then, in the linememory 53, pixel signals d1, d2, d3, d4, and d5 of the pixels 51 in therows 1 to 5 from among the pixels 51 in the row d, are sequentiallytransferred to be read into the outside of the image sensor 32.

0072 AD conversion is performed by the AD converter 52 with respect topixel signals of the pixels 51 in a next row e while the pixel signalsd1 to d5 of the pixels 51 in the rows 1 to 5 from among the pixels 51 inthe row d are being transferred in the line memory 53. Subsequently,pixel signals of the pixels 51 in the columns 1 to 5 from among thepixels 51 in the rows d to h are similarly read into the outside of theimage sensor 32.

0073 As described above, in the image sensor 32, the partial readingmakes it possible to extract a BM image and an RV image from a capturedimage.

0074 Note that, in FIG. 6, a column-parallel-based AD conversionapproach in which a single ADC is responsible for AD conversionperformed on a pixel signal of the pixel 51 in a single column, isadopted as an approach for AD conversion performed by the AD converter52. However, the approach for AD conversion performed by the ADconverter 52 is not limited to the column-parallel-based AD conversionapproach. For example, an area-based AD conversion approach in which asingle ADC is responsible for AD conversion performed on a pixel signalof the pixel 51 in an area that includes a signal pixel or a pluralityof pixels, may be adopted as the approach for AD conversion performed bythe AD converter 52.

<Image That Can be Output by Image Sensor 32>

0075 FIG. 7 illustrates an example of an image that can be output by theimage sensor 32.

0076 Here, a highest-resolution captured image that can be output by theimage sensor 32 is referred to as a highest-resolution image. It isassumed that, for example, the image sensor 32 has the ability to outputa highest-resolution image of a resolution (the number of pixels) Rmaxat (a frame rate of) 60 fps (frame per second) or more.

0077 Here, it is assumed that a resolution RBM for a highest-resolutionBM image (a BM image of a largest number of pixels) that is extractedfrom the highest-resolution image is equal to or less than ½ of theresolution Rmax for the highest-resolution image. It is also assumedthat a resolution RRV for a highest-resolution RV image that isextracted from the highest-resolution image is equal to or less than theresolution RBM for the BM image.

0078 In the present embodiment, it is assumed that, for example, a sumRBM+RRV of the resolution RBM for a BM image and the resolution RRV fora RV image is equal to or less than ½ of the resolution Rmax for thehighest-resolution image. In this case, the use of the image sensor 32capable of outputting a highest-resolution image of the resolution Rmaxat 60 fps (or more) makes it possible to output both the BM image of theresolution RBM and the RV image of the resolution RRV at 120 fps, the BMimage and the RV image being obtained by partially reading thehigh-resolution image.

0079 <Example of Vehicle Transmission Bandwidth That Can be Used forData Transmission in Vehicle 10>

0080 FIG. 8 is a diagram describing an example of a vehicle transmissionbandwidth that can be used for data transmission in the vehicle 10.

0081 In other words, FIG. 8 illustrates examples of a BM image and an RVimage that can be output by the camera unit 11 without data amountsbeing adjusted by the data amount adjuster 33.

0082 For example, the camera unit 11 can output, as a BM image, a colorimage of the resolution RBM in a YUV 4:2:2 format in which the number ofbits per pixel is eight (with respect to each of the brightness and adifference in color). Further, for example, the camera unit 11 canoutput, as an RV image, a color image of the resolution RRV in the YUV4:2:2 format in which the number of bits per pixel is eight.

The BM image of the resolution RBM in the YUV 4:2:2 format in which thenumber of bits per pixel is eight is referred to as a highest-quality BMimage, and the RV image of the resolution RRV in the YUV 4:2:2 format inwhich the number of bits per pixel is eight is referred to as ahighest-quality RV image. 0083

0084 In the present embodiment, it is assumed that the vehicletransmission bandwidth, which is a bandwidth in which data istransmitted from the camera unit 11, is a transmission bandwidth inwhich, for example, two screens of a highest-quality BM image at 60 fpscan be transmitted (in real time). In the present embodiment, thevehicle transmission bandwidth in which two screens of a highest-qualityBM image at 60 fps can be transmitted, makes it possible to transmit,for example, two screens of a highest-quality RV image at 60 fps, sinceRBM RRV. Further, the vehicle transmission bandwidth makes it possibleto transmit, for example, two screens in total that are a single screenof the highest-quality BM image at 60 fps, and a single screen of thehighest-quality RV image at 60 fps.

0085 As described with reference to FIGS. 7 and 8, the camera unit 11 iscapable of outputting both a highest-quality BM image and ahighest-quality RV image at up to 120 fps.

0086 However, in the present embodiment, the vehicle transmissionbandwidth only makes it possible to transmit two screens of ahighest-quality BM image (or RV image) at 60 fps.

0087 The increase in vehicle transmission bandwidth makes it possible totransmit both a highest-quality BM image and a highest-quality RV imageat 120 fps that can be output by the camera unit 11. However, theincrease in vehicle transmission bandwidth results in increasing thecosts for the viewing system.

0088 In the present technology, the camera unit 11 appropriately adjustsdata amounts of a BM image and an RV image to transmit the BM image andthe RV image in the vehicle transmission bandwidth, in order to suppressan increase in the costs for the viewing system.

0089 <Control of Adjustment of Data Amounts of BM Image and RV ImageWhen Two Screens of Highest-Quality BM Image at 60 fps Can beTransmitted in Vehicle Transmission Bandwidth>

0090 FIG. 9 is a diagram describing a first example of a control ofadjustment of data amounts of a BM image and an RV image, the controlbeing performed by the controller 36.

0091 A of FIG. 9 illustrates the adjustment control performed when thevehicle 10 is traveling forward, and is traveling backward at a highspeed that is a speed equal to or greater than a first threshold forspeed.

0092 In this case, the controller 36 performs an adjustment control thatis a control of the data amount adjuster 33, such that a BM image of theresolution RBM at 120 fps is output and output of an RV image isrestricted. In accordance with the adjustment control performed by thecontroller 36, the data amount adjuster 33 adjusts data amounts of a BMimage and an RV image from the image sensor 32 to output the BM image ofthe resolution RBM at 120 fps and to restrict output of the RV image.

0093 Thus, in this case, the BM image of the resolution RBM at 120 fpsis output from the camera unit 11, and the RV image is not output fromthe camera unit 11. Consequently, the BM image of the resolution RBM at120 fps is displayed on the BM display section 21, and the RV image isnot displayed on the RV display section 22.

0094 As described above, when the vehicle 10 is traveling forward, andis traveling backward at the high speed, a driver can confirm a regionsituated further rearward than a region just behind the rear of thevehicle 10, using a BM image of the resolution RBM at 120 fps, that is,a high-resolution BM image at a high frame rate.

0095 Note that, when the vehicle 10 is traveling forward, and istraveling backward at the high speed, an RV image that includes an imageof the region just behind the rear of the vehicle 10 is not displayed.

0096 Further, a BM image of the resolution RBM at 120 fps that is outputby the camera unit 11 can be transmitted in the vehicle transmissionbandwidth in which two screens of a highest-quality BM image (of theresolution RBM) at 60 fps can be transmitted.

0097 Here, examples of a data amount adjusting method for adjusting adata amount of an image that is performed by the data amount adjuster 33include a method for reducing resolution (the number of pixels), amethod for reducing gradation (the number of bits per pixel), a methodfor reducing a frame rate, and a compression method using a specifiedcompression-encoding approach, in addition to the method for restrictingoutput of an image (not outputting an image) as described above.

0098 In addition to being performed by the data amount adjuster 33, arestriction of output of an image, a reduction in resolution, areduction in gradation, and a reduction in frame rate from among thedata amount adjusting methods can be performed by the image sensor 32 bythe controller 36 performing control such as a control of extractionperformed by the image sensor 32.

0099 In other words, the restriction of output of an image such as arestriction of output of an RV image can be performed by the controller36 controlling reading of data from the image sensor 32 and not readinga pixel signal corresponding to the RV image from the pixel 51 (FIG. 6),such that extraction of an RV image from a captured image is restricted.When the vehicle 10 is traveling forward, and is traveling backward atthe high speed, the controller 36 can control, according to vehicleinformation, extraction performed by the image sensor 32 such thatextraction of an RV image from a captured image is restricted. Ofcourse, output of an RV image may be restricted by the data amountadjuster 33, not by the image sensor 32.

0100 The resolution of an image can be reduced by, for example, thecontroller 36 controlling the image sensor 32 such that the number ofpixels 51 from which a pixel signal is read is reduced, or such thatbinning for adding pixel signals of a plurality of pixels 51 isperformed by performing, for example, a so-called source follower (SF)addition or floating diffusion (FD) addition.

0101 The gradation can be reduced by, for example, the controller 36controlling the image sensor 32 such that the number of bits necessaryfor AD conversion performed by the AD converter 52 (FIG. 6) is reduced.

0102 The frame rate can be reduced by, for example, the controller 36controlling the image sensor 32 such that a rate at which a pixel signalis read from the pixel 51, or a rate at which AD conversion is performedby the AD converter 52 is reduced.

0103 B of FIG. 9 illustrates the adjustment control performed when thevehicle 10 is traveling backward at a medium speed that is a speed thatis less than the first threshold and is equal to or greater than asecond threshold less than the first threshold.

0104 In this case, the controller 36 performs an adjustment control thatis a control of the data amount adjuster 133 such that a BM image of aresolution RBMM at 120 fps is output and an RV image of a resolutionRRVM at 30 fps is output, the resolution RBMM being less than theresolution RBM, the resolution RRVM being less than the resolution RRV.In accordance with the adjustment control performed by the controller36, the data amount adjuster 33 adjusts data amounts of a BM image andan RV image from the image sensor 32 to output the BM image of theresolution RBMM at 120 fps and to output the RV image of the resolutionRRVM at 30 fps.

0105 Thus, in this case, the BM image of the resolution RBMM at 120 fpsand the RV image of the resolution RRVM at 30 fps are output by thecamera unit 11. Consequently, the BM image of the resolution RBMM at 120fps is displayed on the BM display section 21, and the RV image of theresolution RRVM at 30 fps is displayed on the RV display section 22.

0106 As described above, when the vehicle 10 is traveling backward atthe medium speed, a driver can confirm a region situated furtherrearward than a region just behind the rear of the vehicle 10, using aBM image of the resolution RBMM at 120 fps, that is, a medium-resolutionBM image at the high frame rate. Further, the driver can confirm theregion just behind the rear of the vehicle 10, using an RV image of theresolution RRVM at 30 fps, that is, a medium-resolution RV image at alow frame rate.

0107 Note that it is assumed that a transmission bandwidth necessary totransmit a RV image of the resolution RRVM at 30 fps is equal to or lessthan a transmission bandwidth for a difference between a maximumtransmission rate (here, a transmission rate necessary to transmit a BMimage of the resolution RBM at 120 fps without compressing the BM image)and a transmission rate (a first transmission rate) necessary totransmit the resolution RBMM at 120 fps. In this case, both a BM imageof the resolution RBMM at 120 fps and an RV image of the resolution RRVMat 30 fps that are output by the camera unit 11 can be transmitted inthe vehicle transmission bandwidth in which two screens of ahighest-quality BM image at 60 fps can be transmitted.

0108 Here, when the transmission bandwidth necessary to transmit an RVimage of the resolution RRVM at 30 fps is not equal to or less than thetransmission bandwidth for the difference between the maximumtransmission rate and the first transmission rate, one of the BM imageand the RV image or both of them can be compressed (compression-encoded)by the data amount adjuster 33 such that the transmission bandwidthnecessary to transmit the RV image of the resolution RRVM at 30 fps isequal to or less than the transmission bandwidth for the differencebetween the maximum transmission rate and the first transmission rate(such that the RV image of the resolution RRVM can be transmitted). Forexample, with respect to a BM image of the resolution RRVM at 120 fps, aportion of or all of the BM image is compressed (compression-encoded).This results in being able to reduce a data amount of the BM image. Withrespect to an RV image of the resolution RRVM at 30 fps, the RV image iscompression-encoded in a state of remaining a color image, or isconverted into a black-and-white image to be compression-encoded. Thisresults in being able to reduce a data amount of the RV image.

0109 C of FIG. 9 illustrates the adjustment control performed when thevehicle 10 is traveling backward at a low speed that is a speed that isless than the second threshold less than the first threshold.

0110 In this case, the controller 36 performs an adjustment control thatis a control of the data amount adjuster 133 such that a BM image of theresolution RBM at 60 fps is output and an RV image of the resolution RRVat 60 fps is output. In accordance with the adjustment control performedby the controller 36, the data amount adjuster 33 adjusts data amountsof a BM image and an RV image from the image sensor 32 to output the BMimage of the resolution RBM at 60 fps and to output the RV image of theresolution RRV at 60 fps.

0111 Thus, in this case, the BM image of the resolution RBM at 60 fpsand the RV image of the resolution RRV at 60 fps are output by thecamera unit 11. Consequently, the BM image of the resolution RBM at 60fps is displayed on the BM display section 21, and the RV image of theresolution RRV at 60 fps is displayed on the RV display section 22.

As described above, when the vehicle 10 is traveling backward at the lowspeed, a driver can confirm a region situated further rearward than aregion just behind the rear of the vehicle 10, using a BM image of theresolution RBM at 60 fps, that is, a high-resolution BM image at amedium frame rate. Further, the driver can confirm the region justbehind the rear of the vehicle 10, using an RV image of the resolutionRRV at 60 fps, that is, a high-resolution RV image at the medium framerate.

0012 The case in which the vehicle 10 is traveling backward at the lowspeed is, for example, a case in which a driver is about to park thevehicle 10, and it is important to confirm a region just behind the rearof the vehicle 10 that is a blind spot as viewed from the driver. Thus,when the vehicle 10 is traveling backward at the low speed, an RV imageis displayed at a higher resolution and at a higher frame rate, comparedto when the vehicle 10 is traveling backward at the high speed or themedium speed. This makes it possible to easily confirm a region of ablind spot and to easily control the vehicle according to a state of theblind spot.

Note that both a BM image of the resolution RBM at 60 fps and an RVimage of the resolution RRV at 60 fps that are output by the camera unit11 can be transmitted in the vehicle transmission bandwidth in which twoscreens of a highest-quality BM image at 60 fps can be transmitted.

0113 <Control of Adjustment of Data Amounts of BM Image and RV ImageWhen Single Screen of Highest-Quality BM Image at 60 fps Can beTransmitted in Vehicle Transmission Bandwidth>

0114 FIG. 10 is a diagram describing a second example of the control ofadjustment of data amounts of a BM image and an RV image, the controlbeing performed by the controller 36.

0115 Here, in the second example of the control of adjustment of dataamounts of a BM image and a RV image, it is assumed that the vehicletransmission bandwidth is a transmission bandwidth in which, forexample, a single screen of a highest-quality BM image (of theresolution RBM) at 60 fps (or more) can be transmitted.

0116 A of FIG. 10 illustrates the adjustment control performed when thevehicle 10 is traveling forward, and is traveling backward at the highspeed that is a speed equal to or greater than the first threshold forspeed.

In this case, the controller 36 performs an adjustment control that is acontrol of the data amount adjuster 33, such that a BM image of theresolution RBM at 60 fps (or more) is output and output of an RV imageis restricted. In accordance with the adjustment control performed bythe controller 36, the data amount adjuster 33 adjusts data amounts of aBM image and an RV image from the image sensor 32 to output the BM imageof the resolution RBM at 60 fps and to restrict output of the RV image.

Thus, in this case, the BM image of the resolution RBM at 60 fps isoutput from the camera unit 11, and the RV image is not output from thecamera unit 11. Consequently, the BM image of the resolution RBM at 60fps is displayed on the BM display section 21, and the RV image is notdisplayed on the RV display section 22.

As described above, when the vehicle 10 is traveling forward, and istraveling backward at the high speed, a driver can confirm a regionsituated further rearward than a region just behind the rear of thevehicle 10, using a BM image of the resolution RBM at 60 fps, that is, ahigh-resolution BM image at the medium frame rate.

Note that, when the vehicle 10 is traveling forward, and is travelingbackward at the high speed, an RV image that includes an image of theregion just behind the rear of the vehicle 10 is not displayed, asdescribed with reference to A of FIG. 9.

Further, a BM image of the resolution RBM at 60 fps that is output bythe camera unit 11 can be transmitted in the vehicle transmissionbandwidth in which a single screen of a highest-quality BM image (of theresolution RBM) at 60 fps can be transmitted.

0117 B of FIG. 10 illustrates the adjustment control performed when thevehicle 10 is traveling backward at the medium speed that is a speedthat is less than the first threshold and is equal to or greater thanthe second threshold less than the first threshold.

In this case, the controller 36 performs an adjustment control that is acontrol of the data amount adjuster 133 such that a BM image of theresolution RBMM at 60 fps is output and an RV image of the resolutionRRVM at 30 fps is output, the resolution RBMM being less than theresolution RBM, the resolution RRVM being less than the resolution RRV.In accordance with the adjustment control performed by the controller36, the data amount adjuster 33 adjusts data amounts of a BM image andan RV image from the image sensor 32 to output the BM image of theresolution RBMM at 60 fps and to output the RV image of the resolutionRRVM at 30 fps.

Thus, in this case, the BM image of the resolution RBMM at 60 fps andthe RV image of the resolution RRVM at 30 fps are output by the cameraunit 11. Consequently, the BM image of the resolution RBMM at 60 fps isdisplayed on the BM display section 21, and the RV image of theresolution RRVM at 30 fps is displayed on the RV display section 22.

As described above, when the vehicle 10 is traveling backward at themedium speed, a driver can confirm a region situated further rearwardthan a region just behind the rear of the vehicle 10, using a BM imageof the resolution RBMM at 60 fps, that is, a medium-resolution BM imageat the medium frame rate. Further, the driver can confirm the regionjust behind the rear of the vehicle 10, using an RV image of theresolution RRVM at 30 fps, that is, a medium-resolution RV image at thelow frame rate.

Note that, when a transmission bandwidth (hereinafter also referred toas a necessary transmission bandwidth) necessary to transmit both a BMimage of the resolution RBMM at 60 fps and an RV image of the resolutionRRVM at 30 fps that are output by the camera unit 11, is not within thevehicle transmission bandwidth in which a single screen of ahighest-quality BM image at 60 fps can be transmitted, the BM image maybe compressed at a first compression rate for medium speed, and the RVimage may be compressed in a state of remaining a color image at asecond compression rate for medium speed that provides a highercompression than the first compression rate for medium speed, or the RVimage may be converted into a black-and-white image to be compressed atthe second compression rate for medium speed, such that the necessarytransmission bandwidth is within the vehicle transmission bandwidth.

0118 C of FIG. 10 illustrates the adjustment control performed when thevehicle 10 is traveling backward at the low speed that is a speed thatis less than the second threshold less than the first threshold.

In this case, the controller 36 performs an adjustment control that is acontrol of the data amount adjuster 133 such that a BM image of aresolution RBML at 60 fps is output and an RV image of the resolutionRRVM at 30 fps is output, the resolution RBML being less than theresolution RBMM. In accordance with the adjustment control performed bythe controller 36, the data amount adjuster 33 adjusts data amounts of aBM image and an RV image from the image sensor 32 to output the BM imageof the resolution RBML at 60 fps and to output the RV image of theresolution RRVM at 30 fps.

Thus, in this case, the BM image of the resolution RBML at 60 fps andthe RV image of the resolution RRV at 30 fps are output by the cameraunit 11. Consequently, the BM image of the resolution RBML at 60 fps isdisplayed on the BM display section 21, and the RV image of theresolution RRVM at 30 fps is displayed on the RV display section 22.

As described above, when the vehicle 10 is traveling backward at the lowspeed, a driver can confirm a region situated further rearward than aregion just behind the rear of the vehicle 10, using a BM image of theresolution RBML at 60 fps, that is, a low-resolution BM image at themedium frame rate. Further, the driver can confirm the region justbehind the rear of the vehicle 10, using an RV image of the resolutionRRVM at 30 fps, that is, a medium-resolution RV image at the low framerate.

Note that, when the necessary transmission bandwidth necessary totransmit both a BM image of the resolution RBML at 60 fps and an RVimage of the resolution RRVM at 30 fps that are output by the cameraunit 11, is not within the vehicle transmission bandwidth in which asingle screen of a highest-quality BM image at 60 fps can betransmitted, the BM image may be compressed at a first compression ratefor low speed, and the RV image may be compressed (in a state ofremaining a color image) at a second compression rate for low speed thatprovides a higher compression than the first compression rate for lowspeed, such that the necessary transmission bandwidth is within thevehicle transmission bandwidth.

0119 Here, a rate that provides a higher compression than the firstcompression rate for medium speed can be adopted as the firstcompression rate for low speed. The same compression rate can be adoptedas the second compression rate for medium speed and the secondcompression rate for low speed. In this case, the following is therelationship among the first and second compression rates for low speed,and the first and second compression rates for medium speed: the secondcompression rate for low speed=the second compression rate for mediumspeed>the first compression rate for low speed>the first compressionrate for medium speed. However, here, it is assumed that a compressionrate exhibiting a larger value provides a higher compression. Thefollowing is the relationship among data amounts respectively obtainedby performing compression at the first compression rate for low speed,by performing compression at the second compression rate for low speed,by performing compression at the first compression rate for mediumspeed, and by performing compression at the second compression rate formedium speed: the data amount obtained by the compression at the secondcompression rate for low speed=the data amount obtained by thecompression at the second compression rate for medium speed<the dataamount obtained by the compression at the first compression rate for lowspeed<the data amount obtained by the compression at the firstcompression for medium speed.

0120 The adjustment control for adjusting data amounts of a BM image andan RV image according to the vehicle speed and the traveling direction(forward or backward) of the vehicle 10 has been described above. Theadjustment control method is not limited to the methods described withreference to FIGS. 9 and 10. In other words, the adjustment controlmethod may be set as appropriate according to, for example, the vehicletransmission bandwidth, the ability of the image sensor 32, and thespecifications of the BM display section 21 and the RV display section22. The data amounts are respectively related to the quality of adisplayed BM image and a displayed RV image. Thus, it can be said thatthe quality of a BM image and the quality of an RV image are changedaccording to the vehicle speed and the traveling direction (forward orbackward) of the vehicle 10.

0121 Further, in the adjustment control, the resolution of the BM imageand the resolution of the RV image can be reduced by reducing the numberof pixels included in the BM image and the RV image, or by(irreversibly) compressing the BM image and the RV image withoutchanging the number of pixels.

0122 When the resolution of a BM image and the resolution of a RV imageare reduced by compressing the BM image and the RV image, it is possibleto adopt, as compression rates at which the BM image and the RV imageare compressed, compression rates such that a BM image and a RV imagethat are obtained by performing compression and decompression and fromwhich a portion of frequency components such as a high frequencycomponent has been lost, each have a substantial resolution (a highestfrequency component) that is equivalent to the resolution (the number ofpixels) described with reference to FIG. 9 or 10.

0123 <Display Processing>

0124 FIG. 11 is a flowchart describing an example of display processingof displaying a BM image and an RV image that is performed in theviewing system of FIG. 3.

0125 In Step S11, the image sensor 32 captures a captured image, and theprocess moves on to Step S12.

0126 In Step S12, the acquisition section 35 acquires vehicleinformation from the vehicle 10, and supplies the vehicle information tothe controller 36. Then, the process moves on to Step S13.

0127 In Step S13, the controller 36 controls extraction performed by theimage sensor 32 according to the vehicle information from theacquisition section 35. The image sensor 32 extracts a BM image and anRV image from the captured image in accordance with the extractioncontrol performed by the controller 36. Then, the image sensor 32supplies the BM image and the RV image to the data amount adjuster 33,and the process moves on to Step S14 from Step S13.

0128 In Step S14, the controller 36 controls adjustment performed by thedata amount adjuster 33 according to the vehicle information from theacquisition section 35. The data amount adjuster 33 adjusts data amountsof the BM image and the RV image from the image sensor 32 in accordancewith the adjustment control performed by the controller 36. Then, thedata amount adjuster 33 supplies, to the output section 34, the BM imageand the RV image of which the respective data amounts have beenadjusted, and the process moves on to Step S15 from Step S14.

0129 In Step S15, the output section 34 outputs, to the outside of thecamera unit 11, the BM image and the RV image that are supplied from thedata amount adjuster 33 and of which the respective data amounts havebeen adjusted, transmits the BM image to the BM display section 21, andtransmits the RV image to the RV display section 22. Then, the processmoves on to Step S16.

0130 In Step S16, the BM display section 21 displays thereon the BMimage from the output section 34 in accordance with the specification ofthe BM display section 21, and the RV display section 22 displaysthereon the RV image from the output section 34 in accordance with thespecification of the RV display section 22.

0131 <Second Example of Configuration of Viewing System>

0132 FIG. 12 is a block diagram illustrating a second example of theconfiguration of the viewing system included in the vehicle 10.

Note that, in the figure, a portion corresponding to that in FIG. 3 isdenoted by the same reference numeral as FIG. 3, and a descriptionthereof is omitted below.

0133 In FIG. 12, the viewing system includes the camera unit 11, the BMdisplay section 21, and the RV display section 22, and the camera unit11 includes the optical system 31, the image sensor 32, the data amountadjuster 33, the output section 34, the acquisition section 35, thecontroller 36, and a memory 71.

Thus, the viewing system of FIG. 12 is similar to the viewing system ofFIG. 3 in including the camera unit 11, the BM display section 21, andthe RV display section 22, and in that the camera unit 11 includes thecomponents from the optical system 31 to the controller 36. 0134

However, the viewing system of FIG. 12 is different from the viewingsystem of FIG. 3 in that the camera unit 11 newly includes the memory71.

0135 In the viewing system of FIG. 12, the image sensor 32 supplies ahighest-resolution image (a highest-resolution captured image) to thememory 71, and the memory 71 stores therein the highest-resolution imagefrom the image sensor 32. Then, the controller 36 performs, as anextraction control, a control of reading of data (a pixel signal) fromthe memory 71 with respect to the memory 71, not with respect to theimage sensor 32. This results in extracting a BM image and an RV imagefrom a highest-resolution image, as in the case of FIG. 3.

0136 The BM-image and the RV-image that are extracted from thehighest-resolution image and stored in the memory 71 are supplied to thedata amount adjuster 33.

0137 Note that, with respect to the restriction of output of an RV imagethat is performed when the vehicle 10 is traveling forward, and istraveling backward at the high speed, as described with reference to Aof FIG. 9 and A of FIG. 10, the restriction of output of an RV image maybe performed in the viewing system of FIG. 12 by restricting extractionof the RV image from a highest-resolution image (by not reading the RVimage) in a control of extraction from the memory 71, in addition tobeing performed by the data amount adjuster 33 restricting output of theRV image.

0138 Further, display processing of displaying a BM image and an RVimage in the viewing system of FIG. 12 is similar to the displayprocessing described with reference to FIG. 11, and thus a descriptionthereof is omitted.

0139 However, in the viewing system of FIG. 12, the image sensor 32captures a captured image and then the captured image is stored in thememory 71 in Step S11. Further, in Step S13, the controller 36 controlsextraction from the memory 71, not from the image sensor 32, accordingto vehicle information from the acquisition section 35. This results inextracting a BM image and an RV image from the captured image stored inthe memory 71.

0140 <Third Example of Configuration of Viewing System>

0141 FIG. 13 is a block diagram illustrating a third example of theconfiguration of the viewing system included in the vehicle 10.

Note that, in the figure, a portion corresponding to that in FIG. 3 isdenoted by the same reference numeral as FIG. 3, and a descriptionthereof is omitted below.

0142 In FIG. 13, the viewing system includes the camera unit 11, the BMdisplay section 21, the RV display section 22, and an extraction section82, and the camera unit 11 includes the optical system 31, the imagesensor 32, the data amount adjuster 33, the output section 34, theacquisition section 35, and a controller 81.

Thus, the viewing system of FIG. 13 is similar to the viewing system ofFIG. 3 in including the camera unit 11, the BM display section 21, andthe RV display section 22, and in that the camera unit 11 includes thecomponents from the optical system 31 to the acquisition section 35. 0143

0144 However, the viewing system of FIG. 13 is different from theviewing system of FIG. 3 in newly including the extraction section 82,and in that the camera unit 11 includes the controller 81 instead of thecontroller 36.

0145 Note that the extraction section 82 may be provided within thecamera unit 11, although the extraction section 82 is provided outsideof the camera unit 11 in FIG. 13.

0146 Vehicle information is supplied by the acquisition section 35 tothe controller 81. Examples of the vehicle information include travelinginformation, the specifications of the BM display section 21 and the RVdisplay section 22, and gyroscopic information. However, in thisexample, the vehicle information supplied by the acquisition section 35does not include positions of the line of sight and the head of adriver. Note that the positions of the line of sight and the head of thedriver of the vehicle 10 are input to the extraction section 82 as aportion of the vehicle information.

0147 As in the case of the controller 36, the controller 81 controlsextraction performed by the image sensor 32 and adjustment performed bythe data amount adjuster 33, according to the vehicle informationsupplied by the acquisition section 35.

0148 However, in the extraction control, the controller 81 causes aregion larger in size than the region R11 to be extracted as a BM imageinstead of controlling (the position of) the region R11 extracted as theBM image, according to one of the positions of the line of sight and thehead of the driver, or according to both of them.

0149 Thus, the BM image output by the output section 34 in FIG. 13 islarger in size than the BM image output by the output section 34 inFIGS. 3 and 12.

0150 In FIG. 13, the BM image larger in size than the region R11 outputby the output section 34 is supplied to the extraction section 82.

0151 The positions of the line of sight and the head of the driver fromamong the vehicle information are supplied to the extraction section 82,in addition to the BM image larger in size than the region R11 beingsupplied by the output section 34 to the extraction section 82.

0152 According to one of the positions of the line of sight and the headof the driver, or according to both of them, the extraction section 82extracts, as a final BM image to be displayed on the BM display section21, a region that is a portion of the BM image larger in size than theregion R11 from the output section 34, that is, a region having the samesize as the region R11, and the extraction section 82 supplies the BMimage to the BM display section 21.

0153 <Control of Extraction of BM Image and RV Image from CapturedImage>

0154 FIG. 14 is a diagram describing an example of a control ofextraction of a BM image and an RV image from a captured image, thecontrol being performed by the controller 81.

0155 As in the case of FIG. 4, FIG. 14 illustrates an image circle ofthe optical system 31 and a captured image captured by the image sensor32 (the light-receiving surface of the image sensor 32).

0156 In the extraction control, the controller 81 controls reading ofdata from the image sensor 32 such that the region R12 is extracted asan RV image, as in the case of the controller 36.

Further, 0157 in the extraction control, the controller 81 controlsreading of data from the image sensor 32 such that not the region R11but a region R31 that is larger in size than the region R11 is extractedfrom the captured image as a BM image.

0158 The region R31 is a region that includes a largest range that adriver can see using an interior rearview mirror by moving his/her lineof sight or his/her head, if the interior rearview mirror is installedin the vehicle 10. The region R11 is a variable region of which theposition is changed according to positions of the line of sight and thehead of the driver, whereas the region R31 is a fixed region.

0159 A region that is situated at a position depending on one of thepositions of the line of sight and the head of the driver or dependingon both of them, and has the same size as the region R11, is extractedby the extraction section 82 from the region R31 described above as afinal BM image to be displayed on the BM display section 21. In otherwords, the region R11 to be observed by the driver using an interiorrearview mirror if the interior rearview mirror is installed in thevehicle 10, is extracted by the extraction section 82 from the regionR31 as a BM image.

0160 <Display Processing>

0161 FIG. 15 is a flowchart describing an example of display processingof displaying a BM image and an RV image that is performed in theviewing system of FIG. 13.

In Step S21, the image sensor 32 captures a captured image, and theprocess moves on to Step S22.

In Step S22, the acquisition section 35 acquires vehicle informationfrom the vehicle 10, and supplies the vehicle information to thecontroller 36. Then, the process moves on to Step S23. 0162

In Step S23, the controller 36 controls extraction performed by theimage sensor 32. As described with reference to FIG. 14, in accordancewith the extraction control performed by the controller 36, the imagesensor 32 extracts, from the captured image, the regions R31 and R12 asa BM image and an RV image, respectively. Then, the image sensor 32supplies the BM image and the RV image to the data amount adjuster 33,and the process moves on to Step S24 from Step S23.

In Step S24, the controller 36 controls adjustment performed by the dataamount adjuster 33 according to the vehicle information from theacquisition section 35. The data amount adjuster 33 adjusts data amountsof the BM image and the RV image from the image sensor 32 in accordancewith the adjustment control performed by the controller 36. Then, thedata amount adjuster 33 supplies, to the output section 34, the BM imageand the RV image of which the respective data amounts have beenadjusted, and the process moves on to Step S25 from Step S24.

In Step S25, the output section 34 outputs, to the outside of the cameraunit 11, the BM image and the RV image that are supplied from the dataamount adjuster 33 and of which the respective data amounts have beenadjusted, and the process moves on to Step S26. Consequently, in FIG.13, 0163 the BM image is supplied to the extraction section 82, and theRV image is transmitted to the RV display section 22.

0164 In Step S26, the extraction section 82 acquires, from the vehicle10, positions of the line of sight and the head of a driver that areincluded in the vehicle information, and the process moves on to StepS27.

0165 In Step S27, the extraction section 82 extracts, from the BM imagefrom the output section 34, a region that is situated at a positiondepending on the positions of the line of sight and the head of thedriver and has the same size as the region R11, the region beingextracted as a final BM image to be displayed on the BM display section21. Then, the extraction section 82 transmits the final BM image to theBM display section 21, and the process moves on to Step S28 from StepS27.

0166 In Step S28, the BM display section 21 displays thereon the BMimage from the extraction section 82 in accordance with thespecification of the BM display section 21, and the RV display section22 displays thereon the RV image from the output section 34 inaccordance with the specification of the RV display section 22.

0167 <Fourth Example of Configuration of Viewing System>

0168 FIG. 16 is a block diagram illustrating a fourth example of theconfiguration of the viewing system included in the vehicle 10.

Note that, in the figure, a portion corresponding to that in FIG. 12 or13 is denoted by the same reference numeral as FIG. 12 or 13, and adescription thereof is omitted below. Examples of vehicle informationacquired by the acquisition section 35 include traveling information,the specifications of the BM display section 21 and the RV displaysection 22, and gyroscopic information. Note that the positions of theline of sight and the head of a driver of the vehicle 10 are input tothe extraction section 82 as a portion of the vehicle information.

0169 In FIG. 16, the viewing system includes the camera unit 11, the BMdisplay section 21, the RV display section 22, and the extractionsection 82, and the camera unit 11 includes the optical system 31, theimage sensor 32, the data amount adjuster 33, the output section 34, theacquisition section 35, the memory, and the controller 81.

Thus, the viewing system of FIG. 16 is similar to the viewing system ofFIG. 13 in including the camera unit 11, the BM display section 21, theRV display section 22, and the extraction section 82, and in that thecamera unit 11 includes the components from the optical system 31 to theacquisition section 35, and the controller 81. 0170

However, the viewing system of FIG. 16 is different from the viewingsystem of FIG. 13 in newly including the memory 71 of FIG. 12.

0171 In the viewing system of FIG. 16, processing similar to theprocessing performed in the viewing system of FIG. 13 is performed,except that the memory 71 stores therein a highest-resolution image (ahighest-resolution captured image) from the image sensor 32 and anextraction control is performed with respect to the memory 71, not withrespect to the image sensor 32, as in the case of the viewing system ofFIG. 12.

0172 <Description of Computer to Which Present Technology is Applied>

0173 Next, the series of processes performed by the controllers 36 and81, the data amount adjuster 33, and the extraction section 82 describedabove can be performed using hardware or software. When the series ofprocesses is performed using software, a program included in thesoftware is installed on, for example, a general-purpose computer.

0174 FIG. 17 is a block diagram illustrating an example of aconfiguration of an embodiment of a computer on which a program used toperform the series of processes described above is installed.

0175 The program can be recorded in advance in a hard disk 905 or a readonly memory (ROM) 903 that is a recording medium included in thecomputer.

0176 Alternatively, the program can be stored (recorded) in a removablerecording medium 911 driven by a drive 909. Such a removable recordingmedium 911 can be provided as so-called package software. Here, examplesof the removable recording medium 911 include a flexible disk, a compactdisc read-only memory (CD-ROM), a magneto-optical (MO) disk, a digitalversatile disc (DVD), a magnetic disk, and a semiconductor memory.

0177 Note that, in addition to being installed on the computer from theremovable recording medium 911 described above, the program can bedownloaded to the computer through a communication network or abroadcast network to be installed on the hard disk 905 included in thecomputer. In other words, for example, the program can be wirelesslytransferred to the computer from a download site through a satellite fordigital satellite broadcasting, or can be transferred to the computer bywire through a network such as a local area network (LAN) or theInternet.

0178 The computer includes a central processing unit (CPU) 902, and aninput/output interface 910 is connected to the CPU 902 through a bus901.

0179 When a command is input by a user operating an input section 907through the input/output interface 910, the CPU 902 executes a programstored in the ROM 903 according to the input command. Alternatively, theCPU 902 loads a program stored in the hard disk 905 into a random accessmemory (RAM) 904 and executes the program.

0180 This results in the CPU 902 performing the processing according tothe flowcharts described above or the processing performed on the basisof the configurations of the block diagrams described above. Then, forexample, as necessary, the CPU 902 outputs a result of the processingusing an output section 906 or transmits the processing result using acommunication section 908 through the input/output interface 910, andthe CPU 902 further records the processing result in the hard disk 905.

0181 Note that the input section 907 includes, for example, a keyboard,a mouse, and a microphone. Further, the output section 906 includes, forexample, a liquid crystal display (LCD) and a speaker.

0182 Here, in the specification, the processes performed by a computerin accordance with a program does not necessarily have to bechronologically performed in the order of the descriptions in theflowcharts. In other words, the processes performed by a computer inaccordance with a program include processes performed in parallel orindividually (for example, parallel processing or processing performedusing an object).

0183 Further, the program may be a program on which processing isperformed by a single computer (processor) or may be a program on whichdistributed-processing is performed by a plurality of computers.Furthermore, the program may be transferred to a remote computer to beexecuted by the remote computer.

Further, the system as used herein refers to a collection of a pluralityof components (such as apparatuses and modules (parts)) and it does notmatter whether all of the components are in a single housing. Thus, aplurality of apparatuses accommodated in separate housings and connectedto one another via a network, and a single apparatus in which aplurality of modules is accommodated in a single housing are both thesystem.

Note that the embodiment of the present technology is not limited to theexamples described above, and various modifications may be made theretowithout departing from the scope of the present technology.

For example, the present technology may also have a configuration ofcloud computing in which a single function is shared to be cooperativelyprocessed by a plurality of apparatuses via a network.

Further, the respective steps described using the flowcharts describedabove may be shared to be performed by a plurality of apparatuses, inaddition to being performed by a single apparatus.

Moreover, when a single step includes a plurality of processes, theplurality of processes included in the single step may be shared to beperformed by a plurality of apparatuses, in addition to being performedby a single apparatus.

Note that the effects described herein are not limitative but are merelyillustrative, and other effects may be provided.

Note that the present technology may take the following configurations.

-   <1> An image-capturing apparatus, including:    -   an image sensor that performs image-capturing;    -   a controller that performs an extraction control for extracting        a first image and a second image from a captured image captured        by the image sensor, the first image being displayed on a first        display section, the second image being displayed on a second        display section; and    -   a data amount adjuster that adjusts data amounts of the first        image and the second image according to vehicle information        acquired from a vehicle.-   <2> The image-capturing apparatus according to <1>, in which    -   the controller performs a control of reading of data from the        image sensor as the extraction control.-   <3> The image-capturing apparatus according to <1>, in which    -   the controller performs a control of reading of data from a        storage as the extraction control, the storage storing therein        the captured image output by the image sensor.-   <4> The image-capturing apparatus according to any one of <1> to    <3>, in which    -   the controller performs the extraction control according to the        vehicle information.-   <5> The image-capturing apparatus according to <4>, in which    -   the vehicle information includes a traveling direction of the        vehicle, and    -   when the traveling direction of the vehicle is a forward        direction, the controller restricts the extraction of the second        image from the captured image.-   <6> The image-capturing apparatus according to <4>, in which    -   the vehicle information includes a position of a line of sight        or a head of a driver of the vehicle, and    -   according to the position of the line of sight or the head of        the driver of the vehicle, the controller controls a region        extracted from the captured image as the first image.-   <7> The image-capturing apparatus according to any one of <1> to    <6>, in which    -   the data amount adjuster adjusts the data amount of the second        image by restricting output of the second image.-   <8> The image-capturing apparatus according to any one of <1> to    <7>, in which    -   the data amount adjuster adjusts the data amount of the first        image by compressing the first image, or the data amount of the        second image by compressing the second image.-   <9> The image-capturing apparatus according to any one of <1> to    <3>, further including    -   an extraction section that extracts a region that is a portion        of the first image as an image to be displayed on the first        display image, according to a position of a line of sight or a        head of a driver of the vehicle.-   <10> The image-capturing apparatus according to any one of <1> to    <9>, in which    -   the vehicle information includes a traveling direction or a        vehicle speed of the vehicle.-   <11> The image-capturing apparatus according to any one of <1> to    <10>, in which    -   the first display section is a display section that is an        alternative to a Class I mirror.-   <12> The image-capturing apparatus according to any one of <1> to    <11>, in which    -   the first image is an image of a region situated further        rearward than a region just behind the rear of the vehicle.-   <13> The image-capturing apparatus according to any one of <1> to    <12>, in which    -   the second image is an image of a region just behind the rear of        the vehicle.-   <14> An image processing method, including:    -   performing an extraction control for extracting a first image        and a second image from a captured image captured by an image        sensor that performs image-capturing, the first image being        displayed on a first display section, the second image being        displayed on a second display section; and    -   adjusting data amounts of the first image and the second image        according to vehicle information acquired from a vehicle.-   <15> A program that causes a computer to operate as a controller and    a data amount adjuster,    -   the controller performing an extraction control for extracting a        first image and a second image from a captured image captured by        an image sensor that performs image-capturing, the first image        being displayed on a first display section, the second image        being displayed on a second display section,    -   the data amount adjuster adjusting data amounts of the first        image and the second image according to vehicle information        acquired from a vehicle.

REFERENCE SIGNS LIST

-   10 vehicle-   11 camera unit-   21 BM display section-   22 RV display section-   23 in-vehicle camera-   31 optical system-   32 image sensor-   33 data amount adjuster-   34 output section-   35 acquisition section-   36 controller-   51 pixel-   52 AD converter-   53 line memory-   71 memory-   81 controller-   82 extraction section-   901 bus-   902 CPU-   903 ROM-   904 RAM-   905 hard disk-   906 output section-   907 input section-   908 communication section-   909 drive-   910 input/output interface-   911 removable recording medium

1. An image-capturing apparatus, comprising: an image sensor thatperforms image-capturing; a controller that performs an extractioncontrol for extracting a first image and a second image from a capturedimage captured by the image sensor, the first image being displayed on afirst display section, the second image being displayed on a seconddisplay section; and a data amount adjuster that adjusts data amounts ofthe first image and the second image according to vehicle informationacquired from a vehicle.
 2. The image-capturing apparatus according toclaim 1, wherein the controller performs a control of reading of datafrom the image sensor as the extraction control.
 3. The image-capturingapparatus according to claim 1, wherein the controller performs acontrol of reading of data from a storage as the extraction control, thestorage storing therein the captured image output by the image sensor.4. The image-capturing apparatus according to claim 1, wherein thecontroller performs the extraction control according to the vehicleinformation.
 5. The image-capturing apparatus according to claim 4,wherein the vehicle information includes a traveling direction of thevehicle, and when the traveling direction of the vehicle is a forwarddirection, the controller restricts the extraction of the second imagefrom the captured image.
 6. The image-capturing apparatus according toclaim 4, wherein the vehicle information includes a position of a lineof sight or a head of a driver of the vehicle, and according to theposition of the line of sight or the head of the driver of the vehicle,the controller controls a region extracted from the captured image asthe first image.
 7. The image-capturing apparatus according to claim 1,wherein the data amount adjuster adjusts the data amount of the secondimage by restricting output of the second image.
 8. The image-capturingapparatus according to claim 1, wherein the data amount adjuster adjuststhe data amount of the first image by compressing the first image, orthe data amount of the second image by compressing the second image. 9.The image-capturing apparatus according to claim 1, further comprisingan extraction section that extracts a region that is a portion of thefirst image as an image to be displayed on the first display image,according to a position of a line of sight or a head of a driver of thevehicle.
 10. The image-capturing apparatus according to claim 1, whereinthe vehicle information includes a traveling direction or a vehiclespeed of the vehicle.
 11. The image-capturing apparatus according toclaim 1, wherein the first display section is a display section that isan alternative to a Class I mirror.
 12. The image-capturing apparatusaccording to claim 1, wherein the first image is an image of a regionsituated further rearward than a region just behind the rear of thevehicle.
 13. The image-capturing apparatus according to claim 1, whereinthe second image is an image of a region just behind the rear of thevehicle.
 14. An image processing method, comprising: performing anextraction control for extracting a first image and a second image froma captured image captured by an image sensor that performsimage-capturing, the first image being displayed on a first displaysection, the second image being displayed on a second display section;and adjusting data amounts of the first image and the second imageaccording to vehicle information acquired from a vehicle.
 15. A programthat causes a computer to operate as a controller and a data amountadjuster, the controller performing an extraction control for extractinga first image and a second image from a captured image captured by animage sensor that performs image-capturing, the first image beingdisplayed on a first display section, the second image being displayedon a second display section, the data amount adjuster adjusting dataamounts of the first image and the second image according to vehicleinformation acquired from a vehicle.